Conversational interactions are joint and dynamic activities where all participants are engaged in the construction of meaning and in the establishment and maintenance of social relationships. Conversations take place according to a cooperation principle for which participants constantly adjust, accommodate or coordinate their behavior with that of their conversational partner. It has been observed in many studies that, over the course of a conversation, speakers adopt similar facial expressions, body postures and gestures, and adapt their way of speaking to that of their interlocutor at different linguistic levels: semantic, syntactic, phonological, phonetic and prosodic.
Prosodic accommodation relates to how speakers adapt their prosodic characteristics to one another as pitch range, speech rate, voice intensity, intonation, rhythm and voice quality, with conversational partners tending to exhibit similar pitch, intensity, speech rate and timing. Accommodation mechanisms variously described as convergence, alignment, entrainment, synchrony, mimicry and the ‘chameleon effect’ constantly arise in conversations and play an important role in social interaction. They allow individuals to understand each other and participate in expressing and recognizing speakers' emotional states and rapport within the frame of the interaction. If cognitive, psychological or physiological impairments prevent such mechanisms, for instance under circumstances of high-stress environments or a lack of linguistic knowledge, difficulties in interacting may arise and may therefore affect the quality of the interaction, especially in terms of the exchange of information and inter-personal relationships.
US patent publication number US2011/270605, Qin Yong et al, discloses a computer implemented method to perform assessment of rhythm. This disclosure only covers the notion of fluency in a way related to rhythm, which is one of the many components of prosody. The fluency/rhythm assessment is based on rules defined in the disclosure based on a reference speech database. Fluency/rhythm rules include pitch reset, pauses, hesitations and final lengthening (as another hesitation type). Overall, the US publication is directed to assess fluency/rhythm accuracy from a speech input (read speech or spontaneous speech) according to standards (a speech database). It states to cover many languages and speaking styles (reading, spontaneous speech). This US patent publication is not suitable for providing a measure of prosodic accommodation between two or more crew members operating in a team environment.
Being able to communicate efficiently has been acknowledged as a vital skill in many different domains. In particular, crew communication skills have been highlighted as a core challenge in the operation of complex machinery such as aircrafts, maritime vessels and such other, highly-specialized, civilian or military vehicles, as well as the performance of complex tasks in the medical domain.
The ability to work together with partners of a crew has been pointed out as an essential skill to enhance communication and information exchange and to improve crew effectiveness and cohesiveness. In this context, communication skills training programs have become widespread over the past decade, with the military and aviation communities in particular having made large investments in this field for better understanding the benefits derived from improved crew skills.
For instance, with reference to the specific field of aircraft operational training, the technical and operational parameters of aircrafts are well understood and defined: aircrafts contain complex data measuring and processing systems for tracking and recording operational parameters, and check that they are maintained within defined parameters in order to identify possible operational or technical errors. Whilst pilots are required to have the technical knowledge to fly and manage their aircraft, they are also required to have effective communication skills in order to operate safely and efficiently in a mission-critical environment. Pilot error is the cause of the majority of commercial aviation incidents and accidents, thus effective communication skills need to be maintained in flight to ensure that no operational errors can occur can occur due to a lack of understanding or communicative deterioration between a pilot and the rest of the crew, particularly a co-pilot.
Accordingly, numerous training protocols exist to train pilots and crew in the non-technical aspects of flying a plane: Multi Crew Co-operation (MCC), Jet Orientation Course (JOC) and Crew Resource Management (CRM). These training protocols are subjectively assessed by trainers, with no method of objective measurement being currently possible and wherein subjectivity lies in the fact that different trainers and training schools may give different results.
A main difficulty thus lies in both how to evaluate crew communication skills in simulation-based training, and what criteria to evaluate. Some researchers have proposed to evaluate both crew performance and crew effectiveness. The first concept refers to the behavioral cognitive and affective processes which crews engage in for coordinating their interactions toward shared goals. The second concept corresponds to the results, or ‘success’, of the performance. In this dual conceptualization, it is proposed that evaluations should not only capture the outcome of crew performance, but also how the crew achieves the outcome. In terms of crew performance, it specifically deals with how individuals within a crew dynamically adjust, coordinate and align their behavior to one another. Many studies have considered and discussed the importance of good communication skills, in particular in the aviation community, but none have mentioned and raised the importance of prosodic accommodation.
No technologies are believed to exist, which make use of prosodic accommodation data for providing objective ratings of crew, such as pilot crew, communication skills. Whilst voice analysis is a well-known field of audio data processing and research, the primary focus is habitually on the voice of a singular person, with little or no consideration given to how the respective voices of two or more crew members talking may change and adapt in relation to each other over the course of a conversation, particularly during the prosecution of a shared objective or task. Moreover, current CRM techniques do not provide a detailed temporal breakdown of the training outcome and results; rather, a general, and thus more subjective and imprecise, overview is given to the trainees, rendering their learning and progress more difficult. Currently, communication skills are subjectively evaluated by human experts. Human evaluation is in nature subjective and therefore may be challenged by many factors (e.g. trainer's fatigue, perception of the trainee) that an objective evaluation is impervious to. The changes in the prosody of two or more interlocutors is often subtle, and dynamic, occurring, to various degrees over the course of the interaction, and It is not possible for a human to accurately recognise and process these discrete changes for any prolonged period of time. At most, humans can recognise individual moments of large scale prosodic changes (e.g. a large change in pitch or intensity, or a large change in the articulation rate).
A new technical method and system for objectively assessing communication skills of crew members is thus proposed, based on prosodic accommodation levels and which mitigates the risk of at least the above problems.